The present invention relates to a method and an apparatus for coding a moving image and to a recording medium for recording a program of coding a moving image, and more particularly to a method and an apparatus for high rate compression coding of moving image data in TV, animation, color-graphic game and so on, and further, to a recording medium for recording a program of coding used for the same.
Heretofore, a system for inter-frame coding with movement compensation (movement-compensated inter-frame coding) has been known which is capable of information-compressed coding of moving-image signals, such as in TV, at high efficiency. FIG. 1 illustrates a prior art, in which the construction of a conventional system for movement-compensated inter-frame coding is shown.
In FIG. 1, video-buffer 51 stores sequentially current image data VD inputted. Meanwhile, frame memory 57 stores preceding image data FD, of one frame earlier, reproduced through decoding. The preceding image data FD is read out for inter-frame expectation coding of current image data VD with the movement of image compensated. In more detail, movement-vector calculation unit 58 calculates the moving of the image (pixel block) between the frames by block-matching search calculation to output the optimum movement vector MV.
In an example of block-matching search calculation, assuming that pixel data of pixel block BD are Bk(k=1 to 16), pixel data of i-th pixel block RDi of 8xc3x978 pixels in search region R in frame memory 57 are Yi,k(k=1 to 16), differential absolute value sum Si between the both pixel blocks Si is calculated by block-matching calculation:
Si=xcexa3∥Bkxe2x88x92Yi,k∥(k=1 to 16)
and then the optimum pixel block Yi,k that makes differential absolute value sum Si minimal is found out, thereby the optimum movement vector MV is obtained.
Meanwhile, variable delay buffer 59 serves to extract pixel block Yi,k corresponding to (optimum) movement vector MV from preceding image data FD to make expected block data PD for movement compensation. Further, subtracter 52 subtracts from respective pixel data VD to be coded the corresponding expected data for movement compensation PD to produce residual difference data PE. Quantumizer 53 quantumizes residual difference data PE to produce coding data CE to be transmitted.
In this state, dequantumizer 54 dequantumizes coding data CE, producing residual difference data PExe2x80x2. Adder 55 adds expected data PD for movement compensation stated above to residual difference data PExe2x80x2, to reproduce current pixel data VDxe2x80x2. Frame buffer 56 accumulates in sequence current pixel data VDxe2x80x2 thus reproduced. After the data for one frame are accumulated, the reproduced data for the one frame are transferred to frame memory 57 as image data FD for the preceding frame.
But the method above in which residual difference data PE for each pixel are quantumized cannot reduce the redundant information associated with the original image itself, thus, high rate of data compression cannot be expected. In this respect, MPEG (Moving Picture Experts Group) system which is popular among the recent systems of moving image compression has achieved a relatively high ratio of data compression by performing inter-frame expectation with movement compensation in a block of 16xc3x9716 image elements (pixels), performing two-dimensional DCT (Discrete Cosine Transform) in a unit of 8xc3x978 pixels related to the expected residual difference thus obtained, quantumizing the sequency thus obtained and performing Haffmann coding.
The predominant portion of the expected residual difference by inter-frame expectation for movement compensation of this kind, however, tends to be concentrated to the peripheral portion of the original image block. This tendency is observed significantly particularly when the original image consists of animation image or color graphic game image containing flat portions and peripheral portions with steep gradient. Therefore, if expected residual difference is developed immediately by a system-fixed orthogonal base system (DCT) such as in conventional MPEG system above, many development coefficients (sequency) containing lower and higher frequency components are required, and a high ratio of data compression cannot be obtained. Further, if high-frequency components are quantumized with low precision in order to elevate data compression ratio, not only image information of the peripheral portions is lost, but also image quality is deteriorated by mosquito noises generated in the peripheral portions.
Accordingly, it is an object of the invention to provide a method for coding of a moving image capable of obtaining high image quality and high ratio of data compression (coding efficiency).
It is another object of the invention to provide an apparatus for coding of a moving image capable of obtaining high image quality and high coding efficiency.
It is still another object of the invention to provide a recording medium for recording a program of coding a moving image in high quality and with high efficiency.
According to the first feature of the invention, the object of the invention stated above is accomplished by a method of coding a moving image comprising the steps of:
comparing a pixel block of current image to be coded and a pixel block of preceding image within a predetermined region of preceding frame, one after another;
seeking a specific preceding-image pixel block minimizing matching error;
if said matching error relative to said specific preceding-image pixel block exceeds an acceptable value, obtaining one or more of orthogonal base systems for approximating AC component vector in said current-image pixel block by inter-frame adaptive orthogonal transformation having, as nest thereof, preceding image data within a predetermined region including said specific preceding image pixel block; and
thereby coding said image data.
According to the first feature of the invention, a high image quality and a high ratio of data compression are obtained by the constitution in which current pixel block B having a matching error relative to the image data of the preceding frame exceeding an acceptable value is approximated by one or more orthogonal base systems by inter-frame adaptive orthogonal transformation utilizing the image data of the preceding frame of a moving image which enable generally to obtain a high correlation between the frames. Further, current pixel block B (alternating current component vector) can be coded with improved efficiency by a small number of orthogonal bases, owing to the constitution utilizing the preceding image data within a predetermined region N including the specific preceding pixel block F minimizing the matching error relative to the current pixel block B as the nest of inter-frame adaptive orthogonal transformation (corresponding to the code book of vector quantumization). Further, remarkable improvement of the coding efficiency (reduction in the amount of codes) can be expected with the original image quality maintained honestly, particularly when the original image consists of animation image or color graphic game image containing a lot of flat portions and peripheral portions with steep gradient.
In the invention, xe2x80x9cpreceding framexe2x80x9d may be a frame immediately before a current frame in the order of displaying a moving image, or a frame which is displayed after the current frame in the order of displaying the moving image, and image data of which is prepared in advance of the display of the current frame for reference of the coding in the current frame.
According to the second feature of the invention, the first basis for approximating AC component vector  less than B greater than  of the current pixel block is produced based on AC component vector  less than F greater than  of the specific preceding pixel block, according to the first feature of the invention.
As specific preceding pixel block F minimizes the matching error relative to current pixel block B according to the first feature of the invention, efficient approximation is accomplished by the constitution of the second invention in which AC component vector  less than F greater than  of the preceding pixel block F is adopted (utilized) as the first base for approximating AC component vector  less than B greater than  of the current pixel block, primarily eliminating the necessity of calculation burden from searching the base. Further, taking advantage of self-similarity of the original image (preceding image), the second or following orthogonal base system can be easily formed for approximating the residual difference vector which concentrates the components to the peripheral portion of the current image, reduction in the number of the bases required as the whole being expected.
According to the third feature of the invention, assuming that the orthogonal base system for approximating AC component vector  less than B greater than  of the current pixel block in the second invention is represented by the linear combination,
xcex11 less than Vxe2x80x21 greater than +xcex12 less than Vxe2x80x22 greater than + . . . +xcex1nk less than Vxe2x80x2nk greater than ,
of the respective normalized orthogonal bases  less than Vxe2x80x2q greater than  based on AC component vectors  less than Uq greater than  of nest pixel block Uq, nk in number, including specific preceding pixel block F, the combination is transformed to the equivalent linear combination:
xcex21 less than U1 greater than +xcex22 less than U2 greater than + . . . +xcex2nk less than Unk greater than ,
which is equivalent to the above combination, utilizing AC-component vectors  less than Uq greater than , while the number of the bases nk, scalar development coefficients xcex2q (q=1 to nk), and the co-ordinate (x,y) of nest pixel block Uq as well as sub-sampling intervals (sx,sy) at least related to AC-component vectors  less than Uq greater than  (q=2 to nk) are coded.
In the method according to the third feature of the invention, taking advantage of the constitution of the invention in which linear combination:
xcex21 less than U1 greater than +xcex22 less than U2 greater than + . . . +xcex2nk less than Unk greater than ,
making use of AC-component vector  less than Uq greater than  of nest pixel block Uq consisting of nk pixels (nkxe2x89xa61) for approximating AC-component vector  less than B greater than  of the current pixel block is obtained finally, and the number of the bases nk, scalar development coefficients xcex2q(q=1 to nk), and the co-ordinate (x,y) of nest pixel block Uq as well as sub-sampling interval (sx,sy) at least related to AC-component vectors  less than Uq greater than (q=2 to nk) are coded, thereby a decoded image is easily obtained by decoding the codes by means of product sum calculation:
xcex21 less than U1 greater than +xcex22 less than U2 greater than + . . . +xcex2nk less than Unk greater than ,
with respect to AC-component vectors  less than Uq greater than  of nest pixel block Uq which is a non-orthogonal base system. Therefore, decoded images can be reproduced at a high speed (real time) with no burden to CPU, memory and so on of a machine like game-machine generally restricted from costs. This method for coding moving images much contributes to reproduction of moving images.
According to the fourth feature of the invention, current pixel block B itself is coded if the number of bases nk in the third invention exceeds a predetermined value. xe2x80x9cCurrent pixel block B itselfxe2x80x9d here means the entire information of current pixel block B, in no relation to the method of coding. For example, the entire data of current pixel block B may be coded, or otherwise, current pixel block B may be divided into the average value (DC) of the block and the residual AC component and either is coded. This is the possible case where current image data not present in the preceding image data occur newly. Thus, any scene of moving images can be coded efficiently by the method according to the fourth feature of the invention.
According to the fifth feature of the invention, current pixel block B and specific preceding pixel block F corresponding thereto are either divided into subordinate pixel blocks of the same dimension, each of which is subjected to inter-frame adaptive orthogonal transformation.
In the method according to the fifth feature of the invention, taking advantage of the constitution in which inter-frame adaptive orthogonal transformation is performed for each subordinate pixel block having a small number of image elements (pixels), the burden from the inter-frame adaptive orthogonal transformation for each subordinate pixel block is reduced significantly. Further, the precision of approximation for the entire current pixel block B is improved, owing to the constitution in which approximation is conducted for each subordinate pixel block.
According to the sixth feature of the invention, current pixel block B is coded based on movement vector  less than M greater than  indicating specific preceding pixel block F if the matching error relative to specific preceding pixel block F in the first invention does not exceed an acceptable value. Thus, a high speed coding process for the whole image is expected. It is noted that movement vector  less than M greater than  of specific current pixel block B in the portion free from movement in the moving image is (0,0).
According to the seventh feature of the invention, the apparatus for coding a moving image comprises:
a first memory storing the image data of the present time;
a second memory storing the image data of the preceding frame;
a movement vector-calculating unit for calculating a movement vector indicating such a specific preceding pixel block that minimizes the matching error by comparing sequentially the current pixel block to be coded with the preceding pixel block within a predetermined region of the preceding frame; and
an inter-frame adaptive orthogonal transformation coding unit for calculating one or more orthogonal base systems for approximating AC-component vector of the current pixel block by means of inter-frame adaptive orthogonal transformation having, as the nest, the preceding image data within a predetermined region including the specific preceding pixel block and coding the base systems, if the matching error relative to the specific preceding pixel block exceeds an acceptable value, so as to code these base systems.
Of course, this inter-frame adaptive orthogonal transformation coding unit may involve any of the processing functions defined in the second to fifth feature of the inventions.
According to the eighth feature of the invention, if the matching error relative to the specific preceding pixel block does not exceed the acceptable value, the current pixel block is coded based on the movement vector indicating the specific preceding pixel block, by means of the inter-frame adaptive orthogonal transformation coding unit in the eighth invention.
According to the ninth invention, a decoding unit is further provided for decoding/storing sequentially the pixel block of the present time based on the coding output of the inter-frame adaptive orthogonal transformation coding unit and the image data of the preceding frame in the second memory, in the apparatus according to the eighth or ninth feature of the invention, to transmit these to the second memory as the image data of the preceding frame. According to the tenth feature of the invention, the common preceding image data are used in the moving image-coding apparatus and the moving image-decoding apparatus for coding/decoding and reproduction.
In the recording medium according to the tenth feature of the invention capable of reading out by a computer, a program for executing a processing defined in any of the foregoing inventions, from first to sixth, in the computer is recorded.